Processor and processing method for an image signal, image display apparatus, generation apparatus and generation method for coefficient data used therein, program for executing each of these methods, and computer-readable medium recording the program

ABSTRACT

A class sorting section obtains a class code CL indicating a class to which pixel data y of a target position in an image signal Vb belongs using motion compensation vector information mi stored in a buffer memory in pair with pixel data of an image signal Va corresponding to the pixel data y. An estimated prediction calculation circuit obtains the pixel data y based on an estimation equation, using pixel data xi of a prediction tap and coefficient data Wi read from a coefficient memory. The coefficient data Wi has been obtained beforehand by a learning executed by use of a student signal which corresponds to the image signal Va and contains the same encoded noise as of the image signal Va, and a teacher signal which corresponds to the image signal Vb and contains no encoded signal.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to a processor and processingmethod for an image signal, an image display apparatus, a generationapparatus and generation method for coefficient data used therein, aprogram for executing each of these methods and a computer-readablemedium which records the program.

[0003] 2. Description of Related Art

[0004] As a system for compression-encoding an image signal, there is anencoding system by means of moving picture experts group 2 (MPEG2) usingdiscrete cosine transform (DCT) operation.

[0005] The DCT operation executes discrete cosine transform onto pixelswithin a block, and re-quantizes the coefficient data obtained as aresult of the discrete cosine transform, and then further executesvariable-length encoding onto the re-quantized coefficient data. In manycases, an entropy encoding such as Huffman encoding is used as thevariable-length encoding. The image data is subject to orthogonaltransform, thereby dividing it into a multitude of items of frequencydata ranging from low frequency to high frequency.

[0006] When the divided frequency data is re-quantized, sincelow-frequency data has high importance taking into consideration of thevisual characteristics of human beings, such low-frequency data isquantized in details. Contrarily, since high-frequency data has lowimportance taking into consideration the visual characteristics of humanbeings, such high-frequency data is roughly quantized. In this manner,high image quality is maintained and compression is realized with highefficiency.

[0007] Decoding by use of a conventional DCT operation convertsquantized data for each frequency component into a typical value of itscode, and executes inverse DCT (IDCT) operation for these components soas to obtain reproduced data. At the time of transformation into thetypical value, a quantized step width at the time of encoding is used.

[0008] In addition, the encoding system by means of MPEG2 executesmotion-compensated predictive encoding.

[0009] As described above, the encoding method by means of MPEG by useof DCT operation executes encoding taking into consideration the visualcharacteristics of human beings. In this manner, high quality ismaintained and compression is realized with high efficiency.

[0010] However, since the encoding which executes DCT operation is aprocessing performed in the unit of blocks, there may occur noises in aform of block, that is, block noises (block distortion) as thecompression rate increases. In addition, at a portion having a suddenchange in brightness such as an edge, a mosquito noise resulted fromroughly quantizing high frequency components occurs.

[0011] Such an encoding noise (encoding distortion) may occur not onlyin the encoding system by means of MPEG2, but also in other encodingsystems.

[0012] The encoding system by means of MPEG 2 executesmotion-compensated predictive encoding. As is already known, the encodeddata of MPEG2 is expressed in a hierarchy structure. The hierarchystructure consists of a sequence layer, a group of picture (GOP) layer,a picture layer, a slice layer, a macro-block layer, and a block layer,starting from the high-order layer in this order.

[0013] The group of picture (GOP) layer starts from a GOP header, andnormally consists of 10 to 15 pictures. The front picture is always anintra-picture (I-picture). The encoding structure of MPEG2 includes, inaddition to the I-picture, a predictive-picture (P-picture) and abidirectionally predictive-picture (B-picture).

[0014] The I-picture is an image obtained as a result ofin-frame/in-field encoding, and is encoded independently from otherframes/fields. The P-picture is an image obtained as a result ofintra-frame/intra-field encoding based on a forward prediction from theI-picture and P-picture which are past in terms of time. The B-pictureis an image obtained as a result of intra-frame/intra-field encodingbased on a bidirectional prediction.

[0015] The units of the predictive encoding of MPEG2 are classified intotwo kinds: a frame picture unit and a field picture unit. When the framepicture unit is selected, the frame produced from an interlaced image isused as a picture unit for motion-compensated predictive encoding. Whenthe field picture unit is selected, two most-recently encoded fields areused as a picture unit for motion-compensated predictive encoding.

[0016] The motion-compensated prediction of frame picture can beexecuted by use of any one of three prediction modes, that is, 1) aframe motion-compensated prediction, 2) a field motion-compensatedprediction, and 3) dual-prime prediction. The motion-compensatedprediction of field picture can be executed by use of any one of threeprediction modes, that is, I) field motion-compensated prediction, 2)dual-prime prediction, and 3) 16×8 motion-compensated prediction.

[0017] The motion-compensated predictive encoding subtracts each pixeldata of a reference block which has been motion-compensated based on themotion vector from each pixel data constituting the input image block,and executes DCT operation onto the residual data remaining after thesubtraction. In this case, the motion vector has an accuracy of ½ pixel.

[0018] For this reason, when the motion vector has a component of ½pixel, pixels with integer accuracy are averaged to obtain pixel with anaccuracy of ½ integer, and in turn to obtain a reference block.Therefore, when the motion vector has a component of ½ pixel, each pixeldata in the reference block has a decreased number of high frequencycomponents. The residual data includes information added thereto forcompensating the decreased number of high frequency components. Contraryto this, when the motion vector has no component of 1/2 pixel, theresidual data includes no information added thereto for compensating thedecreased number of high frequency components.

[0019] An objective of the present invention is to satisfactorily reducean encoding noise (encoding distortion) of an image signal obtained as aresult of decoding a motion-compensated predictive encoded-digital imagesignal.

SUMMARY OF THE INVENTION

[0020] According to the present invention, an image signal processingapparatus converts a first image signal including multiple items ofpixel data to a second image signal including multiple items of pixeldata. The first image signal is generated by decoding amotion-compensated predictive encoded-digital image signal. The imagesignal processing apparatus comprises a class detection device fordetecting a class to which pixel data of a target position in the secondimage signal belongs, based on at least motion-compensated predictiveinformation which has been used at the time of obtaining the first imagesignal corresponding to the target position in the second image signal.

[0021] The motion-compensated predictive information includes motioncompensation vector with an accuracy of ½ pixel. The class detectiondevice detects a class differing depending on whether or not the motioncompensation vector has a ½ pixel component.

[0022] As other motion-compensated predictive information, when themotion-compensated predictive encoding is MPEG2 encoding, informationwith a MPEG2 encoding structure (I-picture, P-picture, or B-picture),information of unit for predictive encoding (frame structure and fieldstructure), motion-compensated predictive information on framemotion-compensated prediction, field motion compensation prediction, andthe like are conceivable.

[0023] The image signal processing apparatus also comprises pixel datageneration device for generating pixel data of the target position inthe second image signal in correspondence with the class detected in theclass detection device.

[0024] For example, the pixel data generation device includescoefficient data generator for generating coefficient data used in anestimation equation. The coefficient data corresponds to the classdetected in the class detection device. The pixel data generation deviceincludes data selector for selecting multiple items of pixel datalocated in the vicinity of the target position in the second imagesignal, based on the first image signal. The pixel data generationdevice includes calculator for calculating and obtaining the pixel dataof the target position in the second image signal based on theestimation equation, by use of the coefficient data generated in thecoefficient data generator and the multiple items of pixel data selectedby the data selector.

[0025] Thus, the pixel data of the target position in the second imagesignal is generated in correspondence with the detected class in themanner as described above. For example, coefficient data correspondingto the class is generated and it is used in the estimation equation.Further, the multiple items of pixel data located in the vicinity of thetarget position in the second image signal is selected, based on thefirst image signal. Then, the pixel data of the target position in thesecond image signal is calculated based on the estimation equation, byuse of the coefficient data and the multiple items of pixel data.

[0026] As described above, a class to which the pixel data of the targetposition in the second image signal belongs is detected, based on atleast the motion-compensated predictive information which has been usedat the time of obtaining the pixel data of the first image signal (inputimage signal) corresponding to the target position in the second imagesignal (output image signal). Then, pixel data of the target position inthe output image signal is generated in correspondence with the detectedclass. This allows an encoded noise of the image signal obtained bydecoding the motion-compensated predictive encoded-digital image signalto be satisfactorily reduced.

[0027] According to the present invention, an image signal processingmethod for converting a first image signal including multiple items ofpixel data into a second image signal including multiple items of pixeldata is provided. The first image signal is generated by decodingmotion-compensated predictive encoded-digital image signal.

[0028] The method includes the steps of: detecting a class to whichpixel data of a target position in the second image signal belongs,based on at least motion-compensated predictive information which hasbeen used at the time of obtaining the pixel data of the first imagesignal corresponding to the target position in the second image signal;and generating pixel data of the target position in the second imagesignal in correspondence with the detected class.

[0029] Further, according to the present invention, a program is usedfor allowing a computer to execute the image signal processing method.Further, according to the present invention, a computer-readable mediumrecords the above-described program.

[0030] According to the present invention, an image display apparatusincludes image signal input device for inputting a first image signalincluding multiple items of pixel data. The first image signal isgenerated by decoding a motion-compensated predictive encoded-digitalimage signal. The image display apparatus also includes an image signalprocessing device for converting the first image signal which has beeninput into the image signal input device into a second image signalincluding multiple items of pixel data so as to output the resultantsecond image signal. The image display apparatus also includes imagedisplay device for displaying an image produced by the second imagesignal output by the image signal processing device onto an imagedisplay element. This image signal processing device has the samestructure as of the image signal processing apparatus described above.

[0031] According to the present invention, an apparatus for generatingcoefficient data of an estimation equation to be used at the time ofconverting a first image signal including multiple items of pixel datainto a second image signal including multiple items of pixel data isprovided. The first image signal is generated by decoding amotion-compensated predictive encoded-digital image signal. Theapparatus includes a decoder for decoding digital signal obtained as aresult of encoding a teacher signal corresponding to the second signalso as to obtain a student signal corresponding to the first imagesignal. The apparatus also includes a class detector for detecting aclass to which pixel data of a target position in the teacher signalbelongs, based on at least the motion-compensated predictive informationwhich has been used at the time of obtaining the pixel data of thestudent signal corresponding to the target position in the teachersignal. The apparatus further includes a data selector for selectingmultiple items of pixel data located in the vicinity of the targetposition in the teacher signal, based on the student signal output fromthe decoder. Additionally, the apparatus includes a calculator forperforming a calculation using the class detected in the class detector,the multiple items of pixel data selected by the data selector, and thepixel data of the target position in the teacher signal and obtainingthe coefficient data for each class.

[0032] Further, according to the present invention, a method forgenerating coefficient data of an estimation equation to be used at thetime of converting a first image signal including multiple items ofpixel data into a second image signal including multiple items of pixeldata is provided. The first image signal is generated by decoding amotion-compensated predictive encoded-digital image signal.

[0033] The method comprises: a first step for decoding digital signalobtained as a result of encoding a teacher signal corresponding to thesecond image signal so as to obtain a student signal corresponding tothe first image signal; a second step for detecting a class to whichpixel data of a target position in the teacher signal belongs, based onat least the motion-compensated predictive information which has beenused at the time of obtaining the pixel data of the student signalcorresponding to the target position in the teacher signal; a third stepfor selecting multiple items of pixel data located in the vicinity ofthe target position in the teacher signal, based on the student signalobtained in the first step; and a fourth step for obtaining thecoefficient data for each class, by use of the class detected in thesecond step, the multiple items of pixel data selected in the thirdstep, and the pixel data of the target position in the teacher signal.

[0034] Further, according to the present invention, a program is usedfor allowing a computer to execute the coefficient data generationmethod mentioned above. Further, according to the present invention, acomputer-readable medium records the above-described program.

[0035] In the present invention, the first pixel signal includingmultiple items of pixel data is generated by decoding themotion-compensated predictive encoded-digital image signal. In thepresent invention, coefficient data of an estimation equation to be usedat the time of converting the first image signal into the second imagesignal including multiple items of pixel data is generated.

[0036] The digital image signal which has been obtained by encoding theteacher signal corresponding to the second image signal is furtherdecoded so as to generate the student signal corresponding to the firstimage signal. A class to which pixel data of the target position in theteacher signal is detected, based on at least the motion-compensatedpredictive information which has been used at the time of obtaining thepixel data of the student signal corresponding to the target position inthe teacher signal.

[0037] Further, the multiple items of pixel data located in the vicinityof the target position in the teacher signal is selected based on thestudent signal. Then, coefficient data is obtained for each class, byuse of the class to which the pixel data of the target position in theteacher signal belongs, the selected multiple items of pixel data, andthe pixel data of the target position in the teacher signal.

[0038] In the manner as described above, the coefficient data of theestimation equation to be used at the time of converting the first imagesignal into the second image signal is generated. At the time ofconverting the first image signal into the second image signal, thepixel data of the target position in the second image signal iscalculated based on the estimation equation, by selectively using thecoefficient data corresponding to the class to which the pixel data ofthe target position in the second image signal belongs.

[0039] As a result, when the first image signal is converted into thesecond image signal by use of the estimation equation, it is possible tosatisfactorily reduce the encoding noise of the image signal obtained bydecoding the motion-compensated predictive encoded-digital image signal.

[0040] The concluding portion of this specification particularly pointsout and directly claims the subject matter of the present invention.However those skill in the art will best understand both theorganization and method of operation of the invention, together withfurther advantages and objects thereof, by reading the remainingportions of the specification in view of the accompanying drawing(s)wherein like reference characters refer to like elements.

BRIEF DESCRIPTION OF THE DRAWINGS

[0041]FIG. 1 is a block diagram showing a constitution of adigital-broadcasting receiver as an embodiment according to the presentinvention;

[0042]FIG. 2 is a block diagram showing a constitution of a MPEG2decoder;

[0043]FIG. 3 is a block diagram showing a class sorting section;

[0044]FIG. 4 is a diagram showing a block for tap selection;

[0045]FIG. 5 is a block diagram showing a constitution of a coefficientdata generation apparatus;

[0046]FIG. 6 is a block diagram showing an exemplary constitution of animage signal processor to be realized in software;

[0047]FIG. 7 is a flowchart showing an image signal processing; and

[0048]FIG. 8 is a flowchart showing a coefficient data generationprocessing.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0049] Hereinafter, embodiments of the present invention will bedescribed with reference to drawings. FIG. 1 shows a constitution of adigital broadcasting receiver 100 as an embodiment according to thepresent invention.

[0050] The digital broadcasting receiver 100 comprises a systemcontroller 101 that includes a microcomputer and controls operations ofthe entire system, and a remote control signal receiving circuit 102 forreceiving a remote control signal. The remote control signal receivingcircuit 102 is connected to the system controller 101, receives a remotecontrol signal RM from a remote control transmitter 200 in response to amanipulation by a user, and supplies a manipulation signal correspondingto the signal RM to the system controller 101.

[0051] The digital broadcasting receiver 100 includes a receivingantenna 105, and a tuner section 106. The tuner section 106 is suppliedwith a broadcasting signal (an RF modulation signal) captured by theantenna 105, and executes selection of stations, demodulation, and errorcorrection and the like so as to obtain a MPEG2 stream as an encodedimage signal in relation to a specified program.

[0052] The digital broadcasting receiver 100 further includes a MPEG2decoder 107 for decoding the MPEG2 stream received from the tunersection 106 so as to obtain an image signal Va, and a buffer memory 108for temporality storing the image signal Va output from the MPEG2decoder 107.

[0053] In this embodiment, the MPEG2 decoder 107 outputs each pixel dataconstituting the image signal Va, motion compensation vector informationmi, which has been used at the time of obtaining the pixel data, andpixel position mode information pi indicating a pixel position where thepixel data is included in any of the 8×8 pixels in the DCT block. Theinformation mi and the information pi are output in pair with each pixeldata. The buffer memory 108 also stores these information mi, pi in pairwith each pixel data.

[0054]FIG. 2 shows a constitution of the MPEG2 decoder 107.

[0055] The decoder 107 includes an input terminal 181 for receiving theinput MPEG2 stream, and a stream buffer 182 for temporarily storing theMPEG2 stream received from the input terminal 181.

[0056] The decoder 107 further includes an extraction circuit 183 and avariable-length decoding circuit 184. The extraction circuit 183extracts a discrete cosine transform (DCT) coefficient as a frequencycoefficient from the MPEG2 stream stored in the stream buffer 182. Thevariable-length decoding circuit 184 executes a variable-length decodingonto a DCT coefficient that has illustratively been Huffman-encoded andhas been extracted in the extraction circuit 183.

[0057] The decoder 107 also includes an extraction circuit 185, aninverse quantization circuit 186, and an inverse DCT circuit 187. Theextraction circuit 185 extracts quantized characteristics specifyinginformation from the MPEG 2 stream stored in the stream buffer 182. Theinverse quantization circuit 186 executes inverse quantization onto thequantized DCT coefficient received from the variable-length decodingcircuit 184, based on the quantized characteristics specifyinginformation extracted in the extraction circuit 185. The inverse DCTcircuit 187 executes an inverse DCT operation onto the DCT coefficientreceived from the inverse quantization circuit 186.

[0058] The decoder 107 also includes a prediction memory circuit 188.The prediction memory circuit 188 stores image signals of anintra-picture (I-picture) and a predictive-picture (P-picture) into itsmemory (not shown). Further, when the inverse DCT circuit 187 outputs animage signal of a P-picture or a bidirectionally predictive-picture(B-picture) using the image signals of the I-picture and the P-picturestored in the unillustrated memory, the prediction memory circuit 188generates and outputs a reference image signal Vref corresponding to theP-picture or the B-picture.

[0059] In addition, the decoder 107 also includes an addition circuit189. When the inverse DCT circuit 187 outputs an image signal of theP-picture or the B-picture, the addition circuit 189 adds the referenceimage signal Vref generated in the prediction memory circuit 188 to thisimage signal. When the inverse DCT circuit 187 outputs an image signalof the I-picture, the prediction memory circuit 188 supplies noreference image signal Vref to the addition circuit 189. In this case,therefore, the addition circuit 189 outputs an image signal of theI-picture received from the inverse DCT circuit 187 as it is.

[0060] The decoder 107 also includes a picture selection circuit 190 andan output terminal 191. The picture selection circuit 190 supplies theprediction memory circuit 188 with image signals of the I-picture andthe P-picture received from the addition circuit 189 to allow its memoryto store these image signals. At the same time, the picture selectioncircuit 190 sorts the image signal of each picture received from theaddition circuit 189 in a correct order and outputs the resultant imagesignals. The imager signal Va is output from the picture selectioncircuit 190 through the output terminal 191.

[0061] The decoder 107 also includes an extraction circuit 192 forextracting encoding control information, that is, picture information PIand motion compensation vector information MI from the MPEG2 streamstored in the stream buffer 182. The motion compensation vectorinformation MI extracted in the extraction circuit 192 is supplied tothe prediction memory circuit 188. The prediction memory circuit 188then executes motion compensation at the time of generating thereference image signal Vref using the motion compensation vectorinformation MI. The picture information PI extracted in the extractioncircuit 192 is supplied to the prediction memory circuit 188 and thepicture selection circuit 190. The prediction memory circuit 188 and thepicture selection circuit 190 respectively identify the picture based onthe picture information PI.

[0062] When the picture selection circuit 190 outputs the image signalVa, it also outputs, in addition to each pixel data constituting thisimage signal Va, the motion compensation vector information mi and thepixel position mode information pi. The motion compensation vectorinformation mi has been used at the time of obtaining the pixel data.The pixel position mode information pi indicates a pixel position wherethe pixel data is included in any of 8×8 pixels in the DCT block. Themotion compensation vector information mi and the pixel position modeinformation pi respectively are output in pair with each pixel data.

[0063] Operations of the MPEG2 decoder 107 shown in FIG. 2 will bedescribed.

[0064] The MPEG2 stream stored in the stream buffer 182 is supplied tothe extraction circuit 183 where a DCT coefficient as a frequencycoefficient is extracted. The variable-length encoded DCT coefficient isthen supplied to the variable-length decoding circuit 184 where it isdecoded. Then, the variable-length decoding circuit 184 supplies thequantized DCT coefficient to the inverse quantization circuit 186 wherethe quantized DCT coefficient is inverse-quantized.

[0065] The inverse DCT circuit 187 executes an inverse DCT operationonto the DCT coefficient received from the inverse quantization circuit186 so as to obtain an image signal of each picture. The image signal ofeach picture is supplied to the picture selection circuit 190 via theaddition circuit 189. In this case, the addition circuit 189 adds thereference image signal Vref received from the prediction memory circuit188 to the image signals of the P-picture and B-picture. The pictureselection circuit 190 sorts image signals of the respective pictures ina correct order and the resultant image signals are output through theoutput terminal 191.

[0066] Returning to FIG. 1, the digital broadcasting receiver 100 alsocomprises an image signal processing section 110 and a display section111. The image signal processing section 110 converts the image signalVa stored in the buffer memory 108 into an image signal Vb having areduced amount of noises as a result of encoding such as block noise(block distortion) and mosquito noise. The display section 111 displaysan image produced by the image signal Vb output from the image signalprocessing section 110. The display section 111 is constituted by adisplay such as a cathode-ray tube (CRT) display, a liquid crystaldisplay (LCD) and the like.

[0067] Operations of the digital broadcasting receiver 100 will bedescribed.

[0068] The MPEG2 stream output from the tuner section 106 is supplied tothe MPEG2 decoder 107 where the MPEG2 stream is decoded. Then, the imagesignal Va output from the decoder 107 is supplied to the buffer memory108 where the image signal Va is temporarily stored.

[0069] The image signal Va stored in the buffer memory 108 in the manneras described above is supplied to the image signal processing section110 where it is converted into an image signal Vb having a reducedamount of encoding noises (encoding distortion). In the image signalprocessing section 110, pixel data constituting the image signal Vb isobtained from pixel data constituting the image signal Va.

[0070] The image signal Vb output from the image signal processingsection 110 is supplied to the display section 111. The display section111 displays an image produced by the image signal Vb on its displayscreen.

[0071] Next, the image signal processing section 110 will be describedmore in detail.

[0072] The image signal processing section 110 includes a tap selectioncircuit 121 as data selection means. The tap selection circuit 121selectively takes out multiple items of pixel data each located in thevicinity of a target position in the image signal Vb from the imagesignal Va stored in the buffer memory 108. Namely, the tap selectioncircuit 121 is used for selectively taking out multiple items of pixeldata for the predictive tap to be used for the prediction. For example,the tap selection circuit 121 takes out multiple items of pixel datawhich include pixel data of the image signal Va corresponding to thepixel data of the target position in the image signal Vb. The multipleitems of pixel data respectively correspond to a DCT block to be a unitfor DCT operation. In this case, when the DCT block consists of 8×8pixels, 64 items of pixel data is taken out.

[0073] The image signal processing section 110 also includes a classsorting section 122 as class detection means for detecting a class towhich the pixel data y of the target position in the image signal Vbbelongs.

[0074] The class sorting section 122 generates the class code CLindicating a class to which the pixel data y of the target position inthe image signal Vb belongs, by use of the multiple items of pixel datalocated in the vicinity of the target position in the image signal Vbamong the multiple items of pixel data constituting the image signal Vastored in the buffer memory 108, the motion compensation vectorinformation mi, and the pixel position mode information pi. The buffermemory 108 stores the information mi and pi in pair with the pixel dataof the image signal Va corresponding to the pixel data of the targetposition in the image signal Vb.

[0075]FIG. 3 shows a constitution of the class sorting section 122.

[0076] The class sorting section 122 includes an input terminal 130A forinputting the image signal Va, tap selection circuits 130B₁ to 130B_(n),and class generation circuits 130C₁ to 130C_(n). The tap selectioncircuits 130B₁ to 130B_(n) selectively take out multiple items of pixeldata for class taps each used for detecting n kinds of classes to whichthe pixel data of the target position in the image signal Vb belongs,based on the image signal Va input into the input terminal 130A. Theclass generation circuits 130C₁ to 130C_(n) generates class codes CL₁ toCL_(n) indicating n kinds of classes using the pixel data taken out inthe tap selection circuits 130B₁ to 130B_(n) respectively.

[0077] In this embodiment, the class codes CL₁ to CL₆ indicating sixkinds of classes are generated. The six kinds of classes are as follows:a space wave class, a time variation class, an AC variation class, aflat class, a line correlation class, and a block edge class. Each ofthe classes will be briefly described.

[0078] 1) The space wave class will be described. It is assumed that thetap selection circuit 130B₁ and the class generation circuit 130C₁constitute a detection system for this space wave class.

[0079] The tap selection circuit 130B, takes out the pixel data in ablock corresponding to the pixel data y of the target position in theimage signal Vb (i.e., the target block shown in FIG. 4) from thecurrent frame of the image signal Va. The class generation circuit 130C₁divides 8×8 items of pixel data into four regions, and calculates apixel average value of each divided region so as to obtain 2×2 items ofhigh-order pixel data. Then, the class generation circuit 130C₁ executesone-bit adaptive dynamic range coding (ADRC) and the like onto each ofthe 2×2 items of pixel data so as to generate 4-bit class code CL₁indicating a space wave class.

[0080] According to the ADRC, a maximum value and a minimum value of themultiple items of pixel data in the class tap are obtained. A dynamicrange, which is a difference between the maximum value and the minimumvalue, is then obtained. Each pixel value is re-quantized incorrespondence with the dynamic range. In the case of 1-bit ADRC, thepixel value is converted into 1 bit depending on whether the pixel valueis larger or smaller than the average value of the multiple items ofpixel values of the class tap. The ADRC is a processing for reducing thenumber of classes indicating the level distribution of the pixel valuesto a relatively small number. Therefore, it is also possible to, insteadof the ADRC, employ encoding for compressing the bit number of pixelvalue such as vector quantization (VQ).

[0081] 2) The time variation class will be described. It is assumed thatthe tap selection circuit 130B₂ and the class generation circuit 130C₂constitute a detection system of the time variation class.

[0082] The tap selection circuit 130B₂ takes out pixel data in a blockcorresponding to the pixel data y of the target position in the imagesignal Vb (i.e., the target block shown in FIG. 4) from the currentframe of the image signal Va. The tap selection circuit 130B₂ also takesout pixel data in a block (i.e., the past block shown in FIG. 4)corresponding to the target block from the past frame preceding thecurrent frame by one frame in the image signal Va.

[0083] The class generation circuit 130C₂ executes subtraction for eachcorresponding pixel between 8×8 items of pixel data in the target blockand 8×8 items of pixel data in the past block so as to obtain adifference value between these 8×8 items of pixel data Then, the classgeneration circuit 130C₂ obtains a sum of squares of the differencevalues between these 8×8 items of pixel data. The class generationcircuit 130C₂ then determines whether or not the sum of squares is athreshold value so as to generate a 2-bit class code CL₂ indicating atime variation class.

[0084] 3) The AC variation class will be described. It is assumed thatthe tap selection circuit 130B₃ and the class generation circuit 130C₃constitute a detection system for the AC variation class.

[0085] The tap selection circuit 1301B₃ takes out pixel data of a blockcorresponding to the pixel data y of the target position in the imagesignal Vb (i.e., the target block shown in FIG. 4) from the currentframe of the image signal Va. The tap selection circuit 130B₃ also takesout pixel data of a block (i.e., the past block shown in FIG. 4)corresponding to the target block from the past frame preceding thecurrent frame by one frame in the image signal Va.

[0086] The class generation circuit 130C₃ executes the DCT operationonto the 8×8 items of pixel data in the target block and the 8×8 itemsof pixel data in the past block respectively so as to obtain a DCTcoefficient (i.e. a frequency coefficient). The class generation circuit130C₃ then obtains the number m₁ of the bottom position where thecoefficient is present at each bottom position in the AC portion, andthe number m₂ of the bottom position where the sign is reversed andeither one of the coefficients is 0. Then, the class generation circuit130C₃ determines whether or not m₁/m₂ is a threshold value so as togenerate a 2-bit class code CL₃ indicating the AC variation class. Inthe block where the time variation is small, it is possible to execute aclass sorting in correspondence with a mosquito distortion by use ofthis AC variation class.

[0087] 4) The flat class will be described. It is assumed that the tapselection circuit 1301B₄ and the class generation circuit 130C₄constitute a detection system for the flat class.

[0088] The tap selection circuit 130B₄ takes out pixel data in a blockcorresponding to the pixel data y of the target position in the imagesignal Vb (i.e., the target block shown in FIG. 4) from the currentframe of the image signal Va. The class generation circuit 130C₄ detectsa maximum value and a minimum value of 8×8 items of pixel data in thetarget block. The class generation circuit 130C₄ then determines whetheror not the dynamic range, which is a difference between the maximumvalue and the minimum value, is a threshold value so as to generate a1-bit class code CL₄ indicating a flat class.

[0089] 5) The line correlation class will be described. It is assumedthat the tap selection circuit 130B₅ and the class generation circuit130C₅ constitute a detection system for this line correlation class.

[0090] The tap selection circuit 130B₅ takes out pixel data in a blockcorresponding to the pixel data y of the target position in the imagesignal Vb (i.e. the target block shown in FIG. 4) from the current frameof the image signal Va.

[0091] The class generation circuit 130C₅ executes subtraction for eachcorresponding pixel between the first line and the second line, thethird line and the fourth line, the fifth line and the six line, and theseventh line and the eighth line of 8×8 items of image data in thetarget block so as to obtain 8×4 items of difference values. Then, theclass generation circuit 130C₅ obtains a sum of squares of the 8×4 itemsof difference values, and then determines whether or not the sum ofsquares is a threshold value so as to generate a 1-bit class code CL₅indicating a line correlation class. This line correlation classindicates whether the in-frame correlation is high as is the case of astatic image, or the motion is rapid and the in-field correlation ishigher than the in-frame correlation.

[0092] 6) The block edge class will be described. It is assumed that thetap selection circuit B₆ and the class generation circuit 130C₆constitute a detection system for the block edge class.

[0093] The tap selection circuit 130B₆ takes out pixel data of a blockcorresponding to the pixel data y of the target position in the imagesignal Vb (i.e. the target block shown in FIG. 4) from the current frameof the image signal Va. The tap selection circuit 130B₆ also takes outpixel data of the blocks horizontally or vertically adjacent the targetblock (i.e. the adjacent blocks shown in FIG. 4) from the current frame.

[0094] The class generation circuit 130C₆ executes subtraction for eachcorresponding pixel between each 8 items of pixel data located alongfour sides of the target block and the pixel data of the adjacentblocks, which is adjacent to the target block, so as to obtain 4×8 itemsof difference values. Then, the class generation circuit 130C₆ obtains asum of squares of difference values as to the respective 8 items ofpixel data, and then determines whether or not the respective 4 items ofthe sums of squares respectively corresponding to the four sides of thetarget block are threshold values so as to generate a 4-bit class codeCL₆ indicating the block edge class.

[0095] The class sorting section 122 includes an input terminal 130D forinputting the motion compensation vector information mi, and a classgeneration circuit 130E for generating a class code CL_(m) indicating asub-pixel class to which the pixel data y of the target position in theimage signal Vb belongs to, based on the motion compensation vectorinformation mi input into the input terminal 130D.

[0096] The class generation circuit 130E determines whether or not themotion compensation vector has been used based on the motioncompensation vector information mi. The class generation circuit 130Edetermines whether or not the motion compensation vector has a ½ pixelcomponent when the motion compensation vector has been used. The classgeneration circuit 130E then sorts the motion compensation vector intoeither one of the three classes so as to generate a two-bit class codeCL_(m) indicating a sub-pixel class.

[0097] The class sorting section 122 includes an input terminal 130F forinputting pixel position mode information pi. The pixel position modeinformation pi is directly used as a class code CL_(p) indicating apixel position mode class. For example, when the DCT block consists of8×8 items of pixel data, this class code CL_(p) is a 6-bit code.

[0098] The class sorting section 122 also includes a class unificationcircuit 130G for unifying the class codes CL₁ to CL_(n) and CL_(m)respectively generated in the class generation circuits 130C₁ to130C_(n) and 130E, and the class code CL_(p) into one class code CL, andan output terminal 130H for outputting the class code CL.

[0099] In this embodiment, the class unification circuit 130G unifiesthe class codes CL₁ to CL₆ and CL_(m) respectively generated in theclass generation circuits 130C, to 130C₆ and 130E, and the class codeCL_(p) into one class code CL.

[0100] If the class codes CL₁ to CL₆, CL_(m), and CL_(p) are simplyunified into one class code, the resultant class code CL shows 16classes (space wave class)×4 classes (time variation class)×4 classes(AC variation class)×2 classes (flat class)×2 classed (line correlationclass)×16 classes (block edge class)×3 classes (sub-pixel class)×64classes (pixel position mode class)=3,145,728 classes.

[0101] In this embodiment, however, the AC variation class is unifiedwith the time variation class as a tree structure. Specifically, whenthe time variation is small, there is a strong likelihood that this is astatic portion. Therefore, a time variation class sorting is firstexecuted, and when the time variation is small, the AC variation classsorting is executed as a tree structure. In this manner, the number ofclasses becomes 7 (=4+4−1) after the time variation class and the ACvariation class have been unified.

[0102] Further, in this embodiment, the line correlation class isunified with the flat class as a tree structure. Specifically, flatclass sorting is first executed, and if not flat, the line correlationclass sorting is executed as a tree structure. In this manner, thenumber of classes becomes 3 (=2+2−1) after the flat class and the linecorrelation class have been unified.

[0103] As a result of executing the class unification by means of a treestructure as described above, the class code CL indicates 16 classes(space wave class)×7 classes (time variation class and the AC variationclass)×16 classes (block edge class)×3 classes (flat class and linecorrelation class)×3 classes (sub-pixel class)×64 classes (pixelposition mode class)=1032192 classes, so that the number of classes canbe significantly reduced.

[0104] Returning to FIG. 1, the image signal processing section 110includes a coefficient memory 123. This coefficient memory 123 storescoefficient data Wi (where i=1 to n, and n represents the number ofpredictive taps) as to each class. The coefficient data Wi is used in anestimation equation to be used in an estimated prediction calculationcircuit 127 described later. This coefficient data Wi is information tobe used for converting the image signal Va into the image signal Vb. Thecoefficient memory 123 receives the class code CL from the class sortingsection 122 described above as address information. The coefficientmemory 123 reads the coefficient data Wi of an estimation equationcorresponding to the class code CL to the estimated predictioncalculation circuit 127. A method for generating the coefficient data Wiwill be described later.

[0105] The image signal processing section 110 includes the estimatedprediction calculation circuit 127 for calculating image data y of thetarget position in the image signal Vb to be produced, by means of thefollowing estimation equation (1), from the pixel data xi of thepredictive tap selectively taken out in the tap selection circuit 121and $\begin{matrix}{y = {\sum\limits_{i = 1}^{n}{W_{i} \cdot x_{i}}}} & (1)\end{matrix}$

[0106] the coefficient data Wi read out of the coefficient memory 123.

[0107] Operations of the image signal processing section 110 will bedescribed.

[0108] The class sorting section 122 generates the class code CLindicating a class to which the pixel data y of the target position inthe image signal Vb belongs, by use of the multiple items of pixel datalocated in the vicinity of the target position in the image signal Vbamong multiple items of pixel data constituting the image signal Vastored in the buffer memory 108, and the motion compensation vectorinformation mi and the pixel position mode information pi stored in thebuffer memory 108 in pair with the pixel data of the image signal Vacorresponding to the pixel data of the target position in the imagesignal Vb.

[0109] Specifically, the class sorting section 122 generates class codesCL₁ to CL₆ respectively indicating the space wave class, the timevariation class, the AC variation class, the flat class, the linecorrelation class, and the block edge class by use of the multiple itemsof pixel data located in the vicinity of the target position in theimage signal Vb. The class sorting section 122 also generates a classcode CL_(m) indicating a sub-pixel class from the motion compensationvector information mi. The class sorting section 122 generates a classcode CL_(p) indicating a pixel position mode class from the pixelposition mode information pi. Finally, the class sorting section 122unifies these class codes CL₁ to CL₆, CL_(m), and CL_(p) into one classcode CL.

[0110] The class code CL thus-generated in the class sorting section 122is supplied to the coefficient memory 123 as read address information.In this manner, the coefficient data Wi corresponding to the class codeCL is read out of the coefficient memory 123. The read coefficient dataWi is supplied to the estimated prediction calculation circuit 127.

[0111] Further, the tap selection circuit 121 selectively takes outmultiple items of pixel data for the predictive tap located in thevicinity of the target position in the image signal Vb from the imagesignal Va stored in the buffer memory 108. In this case, the tapselection circuit 121 takes out multiple items of pixel data, whichincludes pixel data of the image signal Va corresponding to the pixeldata of the target position in the image signal Vb and corresponds tothe DCT block to be used as a unit for DCT operation.

[0112] The estimated prediction calculation circuit 127 obtains thepixel data y of the target position in the image signal Vb to beproduced, based on the estimation equation (1) using the pixel data xiof the predictive tap and the coefficient data Wi read from thecoefficient memory 123.

[0113] As described above, the image signal processing section 110obtains the image signal Vb from the image signal Va using thecoefficient data In this case, the coefficient data obtained from alearning by use of a student signal, which corresponds to the imagesignal Va and contains an encoded noise similar to that of the imagesignal Va, and a teacher signal, which corresponds to the image signalVb and contains no encoded noise, is used as the coefficient data Wi. Inthis manner, the resultant image signal Vb has a significantly reducedamount of encoded noise as compared with the image signal Va.

[0114] The class sorting section 122 in the image signal processingsection 110 generates a class code CL_(m) indicating a sub-pixel classfrom the motion compensation vector information mi as the motioncompensation information, and generates a class code CL including thisclass code CL_(m) unified therewith.

[0115] Therefore, this class code CL shows different classes dependingon the case where the motion compensation vector is not used forobtaining the pixel data of the image signal Va corresponding to thepixel data of the target position in the image signal Vb, the case wherethe motion compensation vector is used therefor but does not contain a½pixel component, and the case where the motion compensation vector isused and contains a ½pixel component.

[0116] The motion compensation vector is used in the case where theimage signal of P-picture or B-picture is decoded. On the other hand,the motion compensation vector is not used in the case where the imagesignal of I-picture is decoded. When the image signal of P-picture orB-picture is decoded by use of the motion compensation vector, thereference image signal Vref is motion-compensated by the motioncompensation vector. Therefore, the state of the image signal Va isdifferent depending on whether the motion compensation vector is used ornot.

[0117] Further, as described above, when the motion compensation vectorcontains a ½pixel component in the MPEG2 encoder, the pixels withinteger accuracy are averaged so as to obtain a pixel with ½ integeraccuracy and then to obtain a reference block. Therefore, when themotion compensation vector contains a ½ pixel component, each pixel dataof the reference image signal Vref has a decreased amount of highfrequency component. The residual data has information added thereto forcompensating the decreased amount of high frequency component. Contraryto this, when the motion compensation vector contains no ½ pixelcomponent, the residual data has no information for compensating thedecreased amount of high frequency component. Therefore, the state ofthe image signal Va is different depending on whether the motioncompensation vector contains a ½ pixel component or not.

[0118] The coefficient data Wi read out of the coefficient memory 123based on the class code CL differs depending on whether the motioncompensation vector is used or not, and further depending on whether themotion compensation vector contains a ½ pixel component or not. Then,the read coefficient data Wi is supplied to the estimated predictioncalculation circuit 127.

[0119] Therefore, the image signal processing section 110 can accuratelyobtain the pixel data y of the target position in the image signal Vbdepending on whether the motion compensation vector is used or not, andfurther depending on whether the motion compensation vector contains a ½pixel component or not. As a result, it is possible to obtain an imagesignal Vb including a satisfactorily reduced amount of encoded noise ascompared with the image signal Va.

[0120] Next, a method for generating coefficient data Wi stored in thecoefficient memory 123 will be described. This coefficient data Wi isgenerated by a learning beforehand.

[0121] First, the learning method will be described. In theabove-mentioned Equation (1), each coefficient data W₁, W₂, . . . , Wnis undefined coefficient before the learning. The learning is executedin each class onto multiple items of signal data. When the number of thelearning data is m, following Equation (2) is set in each class inaccordance with Equation (1). In Equation (2), n denotes the number ofpredictive taps.

Y _(k) =W ₁ ×X _(k1) +W ₂×X_(k2)+ . . . +Wn×X_(kn)  (2)

[0122] (k=1,2, . . . , m)

[0123] When m>n, the coefficient data W₁, W₂, . . . , W_(n) is notuniquely fixed. Therefore, en element e_(k) of an error vector e isdefined in following Equation (3) so as to obtain coefficient data whichrenders the value e² in Equation (4) minimum. Specifically, thecoefficient data is uniquely determined by a least square method.

e _(k) =y _(k) −{W ₁ ×X _(k1) +W ₂ ×X _(k2)+ . . . +W_(n) ×X _(kn)}  (3)

[0124] (k=1,2, . . . , m) $\begin{matrix}{^{2} = {\sum\limits_{k = 1}^{m}e_{k}^{2}}} & (4)\end{matrix}$

[0125] In a practical calculation method for obtaining the coefficientdata, which renders the value of e₂ of Equation (4) minimum, first, thevalue of e₂ is partially differentiated as shown in Equation (5) by thecoefficient data Wi (i=1, 2, . . . , n) so as to obtain the coefficientdata Wi in such a manner that the partial differentiation value becomes0 as to each value of i. $\begin{matrix}{\frac{\partial e^{2}}{\partial{Wi}} = {{\sum\limits_{k = 1}^{m}{2\left( \frac{\partial{ek}}{\partial{Wi}} \right)e_{k}}} = {\sum\limits_{k = 1}^{m}{2{x_{ki} \cdot e_{k}}}}}} & (5)\end{matrix}$

[0126] A specific process for obtaining the coefficient data Wi fromEquation (5) will be described. Defining Xji and Yi as indicated inEquations (6) and (7), Equation (5) can be expressed by the determinantof Equation (8). $\begin{matrix}{X_{ji} = {\sum\limits_{p = 1}^{m}{x_{pi} \cdot x_{pj}}}} & (6) \\{Y_{i} = {\sum\limits_{k = 1}^{m}{x_{ki} \cdot y_{k}}}} & (7) \\{{\begin{bmatrix}X_{11} & X_{12} & \cdots & X_{1n} \\X_{21} & X_{22} & \cdots & X_{2n} \\\cdots & \cdots & \cdots & \cdots \\X_{n1} & X_{n2} & \cdots & X_{nn}\end{bmatrix}\begin{bmatrix}W_{1} \\W_{2} \\\cdots \\W_{n}\end{bmatrix}} = \begin{bmatrix}Y_{1} \\Y_{2} \\\cdots \\Y_{n}\end{bmatrix}} & (8)\end{matrix}$

[0127] Equation (8) is a generally so-called normal equation. Thecoefficient data Wi (i=1, 2, . . . , n) can be obtained by solving thisnormal equation by means of a general solution such as a sweeping method(Gauss-Jordan deletion method).

[0128]FIG. 5 shows a constitution of a coefficient data generationapparatus 150 for generating the coefficient data Wi to be stored in thecoefficient memory 123 of the image signal processing section 110 shownin FIG. 1.

[0129] The coefficient data generation apparatus 150 includes an inputterminal 151, a MPE2 encoder 152, and a MPEG2 decoder 153. The inputterminal 151 receives a teacher signal ST corresponding to the imagesignal Vb. The MPE2 encoder 152 encodes the teacher signal ST to obtaina MPEG2 stream. The MPEG2 decoder decodes the MPEG2 stream to obtain astudent signal SS corresponding to the image signal Va. Here, the MPEG2decoder 153 corresponds to the MPEG2 decoder 107 and the buffer memory108 in the digital broadcasting receiver 100 shown in FIG. 1.

[0130] The coefficient data generation apparatus 150 also includes a tapselection circuit 154 for selectively taking out multiple items of pixeldata located in the vicinity of the target position in the teachersignal ST from the student signal SS received from the MPEG2 decoder 153and outputting the resultant multiple items of pixel data. The tapselection circuit 154 has the same constitution as the tap selectioncircuit 121 of the image signal processing section 110 described above.

[0131] The coefficient data generation apparatus 150 also includes aclass sorting section 155 as class detection means for detecting a classto which the pixel data y of the target position in the teacher signalVb belongs.

[0132] The class sorting section 155 generates a class code CLindicating a class to which the pixel data y of the target position inthe teacher signal ST belongs, by use of multiple items of pixel datalocated in the vicinity of the target position in the teacher signal STamong multiple items of pixel data constituting the student signal SSobtained from the MPEG2 decoder 153, and the motion compensation vectorinformation mi and the pixel position mode information pi, which areobtained from the MPEG2 decoder 153 in pair with the pixel data of thestudent signal SS corresponding to the pixel data of the target positionin the teacher signal ST. The class sorting section 155 has the sameconstitution as the class sorting section 122 of the image signalprocessing section 110 described above.

[0133] The coefficient data generation apparatus 150 includes a delaycircuit 159 for time adjustment of the teacher signal ST supplied to theinput terminal 151; and a normal equation generating section 163. Thenormal equation generating section 163 generates a normal equation (seethe Equation (8) above) for obtaining coefficient data Wi (i=1 to n) foreach class from the pixel data y of each target position obtained fromthe teacher signal ST that has been time-adjusted in the delay circuit159, the pixel data xi of the prediction tap that is selectively takenout in the tap selection circuit 154 in correspondence with the pixeldata y of each target position, and the class code CL generated in theclass sorting section 155 in correspondence with the pixel data y ofeach target position.

[0134] In this case, one learning data is generated in combination withone item of pixel data y and n items of the pixel data xi of predictiontaps corresponding to the one item of pixel data y. A large number ofitems of the learning data are generated for each class between theteacher signal ST and the student signal SS. In this manner, the normalequation generating section 163 generates a normal equation forobtaining coefficient data Wi (i=1 to n) for each class.

[0135] The coefficient data generation apparatus 150 also includes acoefficient data determining section 164 for receiving the data of thenormal equation generated in the normal equation generating section 163and solving the received normal equation to obtain coefficient data Wifor each class, and a coefficient memory 165 for storing thus-obtainedcoefficient data Wi for each class.

[0136] Next, operations of the coefficient data generation apparatus 150shown in FIG. 5 will be described.

[0137] The input terminal 151 receives the teacher signal STcorresponding to the image signal Vb. The MPEG2 encoder 152 then encodesthe teacher signal ST so as to generate the MPEG2 stream. The resultantMPEG2 stream is supplied to the MPEG2 decoder 153. The MPEG2 decoder 153decodes the MPEG2 stream so as to generate the student signal SScorresponding to the image signal Va.

[0138] The class sorting section 155 generates the class code CLindicating a class to which the pixel data y of the target position inthis teacher signal ST belongs, by use of multiple items of pixel datalocated in the vicinity of the target position in the teacher signal STamong multiple items of pixel data constituting the teacher signal SSobtained in the MPEG2 decoder 153, the motion compensation vectorinformation mi and the pixel position mode information pi, which areobtained in the MPEG2 decoder 153 in pair of the pixel data of thestudent signal SS corresponding to the pixel data of the target positionin the teacher signal ST.

[0139] Further, the tap selection circuit 154 selectively takes outmultiple items of pixel data for the prediction tap located in thevicinity of the target position in the teacher signal ST from thestudent signal SS obtained in the MPEG2 decoder 153.

[0140] Then, the normal equation generating section 163 generates thenormal equation (see the Equation (8) above) for obtaining coefficientdata Wi (i=1 to n) for each class, by use of the pixel data y of eachtarget position obtained from the teacher signal ST which has beentime-adjusted in the delay circuit 159, the pixel data xi of theprediction tap selectively taken out in the tap selection circuit 154respectively in correspondence with the pixel data y of each targetposition, and the class code CL generated in the class sorting section155 respectively in correspondence with the pixel data y of each targetposition.

[0141] Then, the coefficient data determining section 164 solves theresultant equation so as to obtain coefficient data Wi for each class.The resultant coefficient data Wi is stored in the coefficient memory165.

[0142] In this manner, in the coefficient data generation apparatus 150shown in FIG. 5, it is possible to generate the coefficient data Wi foreach class to be stored in the coefficient memory 123 of the imagesignal processing section 110 shown in FIG. 1.

[0143] Thus-obtained coefficient data Wi is obtained as a student signalSS as a result of encoding the teacher signal ST so as to generate aMPEG2 stream and then decoding the MPEG2 stream. Thus-obtained studentsignal SS contains an encoded noise as is the case of the image signalVa. Therefore, the image signal Vb obtained by use of this coefficientdata Wi from the image signal Va in the image signal processing section110 shown in FIG. 1 has a reduced amount of encoded noise as comparedwith the image signal Va.

[0144] Alternatively, the processing executed in the image signalprocessing section 110 shown in FIG. 1 may be realized in software by animage signal processing apparatus 300 shown in FIG. 6, for example.

[0145] First, an image signal processing apparatus 300 shown in FIG. 6will be described. The image signal processing apparatus 300 includes aCPU 301 for controlling operations of entire apparatus, a read onlymemory (ROM) 302 for storing a control program for the CPU 301,coefficient data and the like, and a random access memory (RAM) 303constituting a working area for the CPU 301. The CPU 301, the ROM 302and the RAM 303 are respectively connected to a bus 304.

[0146] The image signal processing apparatus 300 also includes a harddisc drive (HDD) 305 as an external storage apparatus, and a drive (FDD)307 for driving a Floppy (Trade Name) disc 306. These drives 305 and 307are respectively connected to the bus 304.

[0147] The image signal processing apparatus 300 also includes acommunication section 308 for communicating with an communicationnetwork 400 such as the Internet and the like in a wired or wirelessmanner. The communication section 308 is connected to the bus 304 via aninterface 309.

[0148] The image signal processing apparatus 300 also includes a userinterface section. This user interface section includes a remote controlsignal receiving circuit 310 for receiving a remote control signal RMfrom a remote control transmitter 200, and a display 311 constituted bya liquid crystal display (LCD) and the like. The receiving circuit 310is connected to the bus 304 via an interface 312, and similarly, thedisplay 311 is connected to the bus 304 via an interface 313.

[0149] The image signal processing apparatus 300 also includes an inputterminal 314 for inputting the image signal Va, and an output terminal315 for outputting the image signal Vb. The input terminal 314 isconnected to the bus 304 via an interface 316, and similarly, the outputterminal 315 is connected to the bus 304 via an interface 317.

[0150] Instead of storing the control program, the coefficient data andthe like into the ROM 302 beforehand in the manner described above, theymay be downloaded from the communication network 400 such as theInternet via the communication section 308 for example. They may be alsostored in a hard disc or the RAM 303 for use. Alternatively, the controlprogram, the coefficient data and the like may be provided in the formof Floppy (Trade Name) disc 306 storing them.

[0151] Further, instead of inputting the image signal Va to be processedfrom the input terminal 314, the image signal Va may be recorded in ahard disc beforehand, or may be downloaded beforehand from thecommunication network 400 such as the Internet via the communicationsection 308. Further, instead of or at the same time of outputting theprocessed image signal Vb through the output terminal 315, the processedimage signal Vb may be supplied to the display 311 to make an imagedisplay. Alternatively, the processed image signal Vb may be also storedin a hard disc or may be sent out to the communication network 400 suchas the Internet via the communication section 308.

[0152] Referring to the flowchart of FIG. 7, a procedure for obtainingan image signal Vb from an image signal Va in the image signalprocessing apparatus 300 shown in FIG. 6 will be described.

[0153] First, a processing is started in Step ST61. Then, in Step S62,an image signal Va by one frame or one field is input, for example, fromthe input terminal 314 to the apparatus. The RAM 303 temporarily storesthe pixel data constituting the image signal Va thus input from theinput terminal 314. If the image signal Va is recorded in the hard discdrive 305 within the apparatus beforehand, this drive 305 reads theimage signal Va, and the RAM 303 then temporality stores the pixel dataconstituting thus-read image signal Va.

[0154] Then, in Step ST63, it is determined whether or not all theframes or all the fields of the image signal Va have been processed.Then, when the processing has been finished, the processing is ended inStep ST64. Contrarily, when the processing has not yet been finished,the procedure proceeds to Step ST65.

[0155] In Step ST65, the class code CL indicating a class to which thepixel data of the target position in the image signal Vb belongs, basedon multiple items of pixel data located in the vicinity of the targetposition in the image signal Vb among the image signal Va, which hasbeen input in Step ST62, and also based on, although not mentionedabove, motion compensation vector information mi and pixel position modeinformation pi, which have been input in pair of the pixel data of theimage signal Va corresponding to the pixel data of the target positionin the image signal Vb.

[0156] Next, in Step ST66, multiple items of pixel data (pixel data forthe prediction tap) located in the vicinity of the target position inthe image signal Vb are obtained from the image signal Va input in StepST62. Then, in Step ST67, it is determined whether or not the processingfor obtaining the pixel data of the image signal Vb has been finished inall the area of the pixel data of the image signal Va by one frame orone field which has been input in Step ST62. When finished, theprocedure returns to Step ST62 and the input of the image signal Va forthe next one frame or one field is started. Contrarily, when notfinished the procedure proceeds to Step ST68.

[0157] In Step ST68, the pixel data of the target position in the imagesignal Vb is generated based on the estimation equation, by use of thecoefficient data Wi corresponding to the class code CL generated in StepST65 and the pixel data for the prediction tap. Subsequently, theprocedure returns to Step ST65 to start a processing for the next targetposition.

[0158] As a result of the processing along the flowchart shown in FIG. 7as described above, the pixel data of the input image signal Va isprocessed, thereby obtaining pixel data of the image signal Vb. Asdescribed above, thus-obtained image signal Vb is output through theoutput terminal 315 or is supplied to the display 311 to create an imagethereon, or is supplied to the hard disc drive 305 so as to be recordedin the hard disc.

[0159] The processing in the coefficient data generation apparatus 150shown in FIG. 5 may be realized in software, although the illustrationof the processing apparatus is omitted.

[0160] Referring to the flowchart of FIG. 8 the procedure for generatingcoefficient data will be described.

[0161] First, a processing is started in Step ST81. Then, a teachersignal ST is input only by one frame or one field in Step ST82. Then, inStep ST83, it is determined whether or not all the frames or all thefields of the teacher signal has been processed. When not processed yet,in Step ST84, a student signal SS is generated from the teacher signalST, which has been input in Step ST82.

[0162] Then, in Step ST85, a class code CL indicating a class to whichpixel data of the target position in the teacher signal ST belongs isgenerated, based on multiple items of pixel data located in the vicinityof the target position in the teacher signal ST among the studentsignals SS generated in Step ST84, and also based on, although notmentioned above, the motion compensation vector information mi and thepixel position mode information pi, which have been obtained incorrespondence with the pixel data of the student signal SScorresponding to the pixel data of the target position in the teachersignal ST.

[0163] Further, in Step ST86, multiple items of pixel data (pixel datafor the prediction tap) located in the vicinity of the target positionin the teacher signal ST are obtained from the student signal SSgenerated in Step ST84. Then, in Step ST87, it is determined whether ornot the learning has been finished in all the area of the pixel data ofthe teacher signal ST by one frame or one field which has been input inStep ST82. When the learning has been finished, the procedure returns toStep ST82 and the input of the teacher signal ST for the next one frameor one field is started to repeat the same processing described above.Contrarily, when the learning has not been finished, the procedureproceeds to Step ST88.

[0164] In Step ST88, a normal equation (see the Equation (8) above) forobtaining coefficient data Wi for each class is generated, by use of theclass code CL generated in Step ST85, multiple items of pixel data xiobtained in Step ST86, and the pixel data y of the target position inthe teacher signal ST. Subsequently, the procedure returns to Step ST85to start a processing for the next target position.

[0165] When the processing is finished in Step ST83 described above, inStep ST89, the normal equation generated in the Step ST88 describedabove is solved by means of a sweeping method and the like so as tocalculate coefficient data of each class. Then, in Step ST90, thecoefficient data of each class is stored in the memory. Subsequently,the processing is ended in Step ST91.

[0166] As a result of the processing along the flowchart shown in FIG. 8as described above, the coefficient data Wi of each class can beobtained in the same manner as of the coefficient data generationapparatus 150 shown in FIG. 5.

[0167] In the embodiment described above, the motion compensation vectorinformation mi is used for class sorting operation. Alternatively, otherkinds of motion compensation predictive information may be used. Forexample, when the motion compensation predictive encoding is MPEG2encoding, it is possible to use information with a MPEG2 encodingstructure (I-picture, P-picture, or B-picture), the unit of predictiveencoding (frame structure, field structure), motion compensationpredictive information (frame motion compensation prediction, fieldmotion compensation prediction, and the like), or the like. In thisalternative case as well, the encoded noise of the image signal can besatisfactorily reduced.

[0168] In the embodiment described above, the MPEG2 stream is handled.However, the present invention may also similarly be applicable to anyother cases where a digital image signal obtained as a result ofmotion-compensated predictive encoding is handled.

[0169] According to the present invention, a class to which the pixeldata of the target position in the output image signal belongs isdetected, based on the motion-compensated predictive information whichhas been used at the time of obtaining the pixel data of the input imagesignal corresponding to the target position in at least the output imagesignal. Then, the pixel data of the target position in the output imagesignal is generated in correspondence with the detected class. In thismanner, it is possible to satisfactorily reduce the encoded noise of theimage signal obtained as a result of decoding the digital image signalwhich has been motion-compensated predictive encoded.

[0170] While the foregoing specification has described preferredembodiment(s) of the present invention, one skilled in the art may makemany modifications to the preferred embodiment without departing fromthe invention in its broader aspects. The appended claims therefore areintended to cover all such modifications as fall within the true scopeand spirit of the invention.

What is claimed is:
 1. An image signal processing apparatus forprocessing a first image signal including multiple items of pixel data,said first image signal being generated by decoding a motion-compensatedpredictive encoded-digital image signal, to allow the first image signalto be converted to a second image signal including multiple items ofpixel data, said apparatus comprising: class detection means fordetecting a class to which pixel data of a target position in saidsecond image signal belongs, based on at least motion-compensatedpredictive information which has been used at the time of obtaining thepixel data of said first image signal corresponding to the targetposition in said second image signal; and pixel data generation meansfor generating pixel data of the target position in said second imagesignal in correspondence with said class detected in said classdetection means.
 2. The image signal processing apparatus according toclaim 1, wherein said pixel data generation means comprises: coefficientdata generation means for generating coefficient data used in anestimation equation, said coefficient data corresponding to the classdetected in said class detection means; data selection means forselecting multiple items of pixel data located in the vicinity of thetarget position in said second image signal, based on said first imagesignal; and calculation means for calculating and obtaining the pixeldata of the target position in said second image signal based on saidestimation equation, by use of the coefficient data generated in saidcoefficient data generation means and the multiple items of pixel dataselected by said data selection means.
 3. The image signal processingapparatus according to claim 1, wherein said motion-compensatedpredictive information includes motion compensation vector informationwith an accuracy of ½ pixel, and said class detection means detects aclass differing depending on whether or not said motion compensationvector has a ½ pixel component.
 4. An image signal processing method forprocessing a first image signal including multiple items of pixel data,said first image signal being generated by decoding a motion-compensatedpredictive encoded-digital image signal, to allow the first image signalto be converted to a second image signal including multiple items ofpixel data, said method comprising the steps of: detecting a class towhich pixel data of a target position in the second image signalbelongs, based on at least motion-compensated predictive informationwhich has been used at the time of obtaining the pixel data of saidfirst image signal corresponding to the target position in said secondimage signal; and generating pixel data of the target position in saidsecond image signal in correspondence with said detected class.
 5. Acomputer-readable medium recording a program of an image signalprocessing method for processing a first image signal including multipleitems of pixel data, said first image signal being generated by decodinga motion-compensated predictive encoded-digital image signal, to allowthe first image signal to be converted to a second image signalincluding multiple items of pixel data, said method comprising the stepsof: detecting a class to which pixel data of a target position in thesecond image signal belongs, based on at least motion-compensatedpredictive information which has been used at the time of obtaining thepixel data of said first image signal corresponding to the targetposition in said second image signal; and generating pixel data of thetarget position in said second image signal in correspondence with saiddetected class.
 6. A program executed by a computer, said programcomprising the steps of: in order to convert a first image signalincluding multiple items of pixel data, said first image signal beinggenerated by decoding a motion-compensated predictive encoded-digitalimage signal, to a second image signal including multiple items of pixeldata, detecting a class to which pixel data of a target position in thesecond image signal belongs, based on at least motion-compensatedpredictive information which has been used at the time of obtaining thepixel data of the first image signal corresponding to the targetposition in said second image signal; and generating pixel data of thetarget position in said second image signal in correspondence with saiddetected class.
 7. An image display apparatus comprising: image signalinput means for inputting a first image signal including multiple itemsof pixel data, said first image signal being generated by detecting amotion-compensated predictive encoded-digital image signal; image signalprocessing means for processing said first image signal thus input bysaid input means to allow the first image signal to be converted to asecond image signal including multiple items of pixel data, andoutputting the resultant second image signal; and image display meansfor displaying an image produced by said second image signal output bythe image signal processing means onto an image display element, whereinsaid image signal processing means comprises: class detection means fordetecting a class to which pixel data of a target position in saidsecond image signal belongs, based on at least motion-compensatedpredictive information which has been used at the time of obtaining thepixel data of said first image signal corresponding to the targetposition in said second image signal; and pixel data generation meansfor generating the pixel data of the target position in said secondimage signal in correspondence with the class detected in said classdetection means.
 8. An apparatus for generating coefficient data of anestimation equation to be used at the time of converting a first imagesignal including multiple items of pixel data, said first image signalbeing generated by decoding a motion-compensated predictiveencoded-digital image signal, to a second image signal includingmultiple items of pixel data, said apparatus comprising: decoding meansfor decoding digital image signal obtained as a result of encoding ateacher signal corresponding to said second image signal and obtaining astudent signal corresponding to said first image signal; class detectionmeans for detecting a class to which pixel data of a target position insaid teacher signal belongs, based on at least the motion-compensatedpredictive information which has been used at the time of obtaining thepixel data of said student signal corresponding to the target positionin said teacher signal; data selection means for selecting multipleitems of pixel data located in the vicinity of the target position insaid teacher signal, based on the student signal output from saiddecoding means; and calculation means for performing a calculation usingthe class detected in said class detection means, the multiple items ofpixel data selected by said data selection means, and the pixel data ofthe target position in said teacher signal, and obtaining thecoefficient data for each class.
 9. The apparatus according to claim 8,wherein said motion-compensated predictive information includes motioncompensation vector information with an accuracy of ½ pixel, and saidclass detection means detects a class differing depending on whether ornot the motion compensation vector has a ½ pixel component.
 10. A methodfor generating coefficient data of an estimation equation to be used atthe time of converting a first image signal including multiple items ofpixel data, said first image signal being generated by decoding amotion-compensated predictive encoded-digital image signal to a secondimage signal including multiple items of pixel data, said methodcomprising: a first step for decoding digital image signal obtained as aresult of encoding a teacher signal corresponding to said second imagesignal and obtaining a student signal corresponding to said first imagesignal; a second step for detecting a class to which pixel data of atarget position in said teacher signal belongs, based on at leastmotion-compensated predictive information which has been used at thetime of obtaining the pixel data of said student signal corresponding tothe target position in said teacher signal; a third step for selectingmultiple items of pixel data located in the vicinity of the targetposition in said teacher signal, based on the student signal obtained insaid first step; and a fourth step for obtaining said coefficient datafor said each class, by use of the class detected in said second step,the multiple items of pixel data selected in said third step, and thepixel data of the target position in said teacher signal.
 11. Acomputer-readable medium recording a program of a method for generatingcoefficient data of an estimation equation to be used at the time ofconverting a first image signal including multiple items of pixel data,said first image signal being generated by decoding a motion-compensatedpredictive encoded-digital image signal, to a second image signalincluding multiple items of pixel data, said method comprising: a firststep for decoding digital image signal obtained as a result of encodinga teacher signal corresponding to said second signal and obtaining astudent signal corresponding to said first image signal; a second stepfor detecting a class to which pixel data of a target position in saidteacher signal belongs, based on at least motion-compensated predictiveinformation which has been used at the time of obtaining the pixel dataof said student signal corresponding to the target position in saidteacher signal; a third step for selecting multiple items of pixel datalocated in the vicinity of the target position in said teacher signal,based on the student signal obtained in said first step; and a fourthstep for obtaining said coefficient data for said each class, by use ofthe class detected in said second step, the multiple items of pixel dataselected in said third step, and the pixel data of the target positionin said teacher signal.
 12. A program executed by a computer, saidprogram comprising: in order to generate coefficient data of anestimation equation to be used at the time of converting a first imagesignal including multiple items of pixel data, said first image signalbeing generated by decoding a motion-compensated predictiveencoded-digital image signal, to a second image signal includingmultiple items of pixel data, a first step for decoding digital imagesignal obtained as a result of encoding a teacher signal correspondingto said second signal and obtaining a student signal corresponding tosaid first image signal; a second step for detecting a class to whichpixel data of a target position in said teacher signal belongs, based onat least motion-compensated predictive information which has been usedat the time of obtaining the pixel data of said student signalcorresponding to the target position in said teacher signal; a thirdstep for selecting multiple items of pixel data located in the vicinityof the target position in said teacher signal, based on the studentsignal obtained in said first step; and a fourth step for obtaining saidcoefficient data for each class, by use of the class detected in saidsecond step, the multiple items of pixel data selected in said thirdstep, and the pixel data of the target position in said teacher signal.